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ABSTRACT. A graphical realization of a linear code C consists of an assignment of the coordi- 
nates of C to the vertices of a graph, along with a specification of linear state spaces and linear 
"local constraint" codes to be associated with the edges and vertices, respectively, of the graph. The 
K-complexity of a graphical realization is defined to be the largest dimension of any of its local 
constraint codes. K-complexity is a reasonable measure of the computational complexity of a sum- 
product decoding algorithm specified by a graphical realization. The main focus of this paper is on 
the following problem: given a linear code C and a graph Q, how small can the /i-complexity of a real- 
ization of C on Q be? As useful tools for attacking this problem, we introduce the Vertex-Cut Bound, 
and the notion of "vc-treewidth" for a graph, which is closely related to the well-known graph- 
theoretic notion of treewidth. Using these tools, we derive tight lower bounds on the K-complexity 
of any realization of C on Q. Our bounds enable us to conclude that good error-correcting codes can 
have low-complexity realizations only on graphs with large vc-treewidth. Along the way, we also 
prove the interesting result that the ratio of the K-complexity of the best conventional trellis realiza- 
tion of a length-n code C to the K-complexity of the best cycle-free realization of C grows at most 
logarithmically with codelength n. Such a logarithmic growth rate is, in fact, achievable. 



1. Introduction 

The study of graphical models of codes and the associated message-passing decoding algorithms 
is a major focus of current research in coding theory. This is attributable to the fact that coding 
schemes using graph-based iterative decoding strategies — e.g., turbo codes and low-density parity 
check (LDPC) codes — have low implementation complexity, while their performance is close to 
the optimum predicted by theory. A unified treatment of graphical models and the associated decod- 
ing algorithms began with the work of Wiberg, Loeliger and Koetter lfl"8l , lfT9| , and has since been 
abstracted and refined under the framework of the generalized distributive law [T], factor graphs 
02), and normal realizations 15), 0. In fact, the study of cycle-free graphical models of codes (i.e., 
models in which the underlying graphs are cycle-free) can be traced back to the introduction of the 
Viterbi decoding algorithm in the 1960's, which led to the study of trellis representations of codes. 
A comprehensive account of the history and development of trellis representations can be found in 
E). 

In this work, we will follow the approach of Forney (6j,(7), and Halford and Chugg (8) in 
studying the general "extractive" problem of constructing low-complexity graphical models for 
a given linear code. Roughly speaking, a low-complexity graphical model is one that implies a 
low-complexity decoding algorithm. In particular, in this paper, we investigate the question of how 
small the complexity of an arbitrary graphical model for a given code can be. 

We briefly introduce graphical models here; a detailed description can be found in Section [2] A 
graph decomposition of a code C is a mapping of the set of coordinates of C to the set of vertices of a 
graph. A graph decomposition may be viewed as an assignment of symbol variables to the vertices 
of the graph. A graph decomposition can be extended to a graphical model which additionally 
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assigns state variables to the edges of the graph, and specifies a local constraint code at each vertex 
of the graph. The full behavior of the model is the set of all configurations of symbol and state 
variables that satisfy all the local constraints. Such a model is called a graphical realization of C if 
the restriction of the full behavior to the set of symbol variables is precisely C. The realization is 
said to be cycle-free if the underlying graph in the model has no cycles. A trellis representation of 
a code can be viewed as a cycle-free realization in which the underlying graph is a simple path. In 
contrast, a tailbiting trellis representation lfQl . lfl4l is a graphical realization in which the underlying 
graph consists of a single cycle. 

We will focus our attention on the case of realizations of linear codes on connected graphs only. 
Indeed, there is no loss of generality in doing so, since a linear code C has a realization on a graph 
Q that is not connected if and only if C can be expressed as the direct sum of codes that may 
be individually realized on the connected components of Q 0. In this context, we will refer to 
cycle-free graphical realizations simply as tree realizations, as the underlying graph is a connected, 
cycle-free graph, i.e., a tree. 

It is by now well known that any graphical realization of a code specifies a canonical iterative 
message-passing decoding algorithm, namely, the sum-product algorithm, on the underlying graph 
[ 1 ],[6],[ 12l. lfT8l . When the underlying graph is a tree, the sum-product algorithm provides an exact 
implementation of maximum-likelihood (ML) decoding. Even when the underlying graph contains 
cycles, empirical evidence suggests that, in many cases, the sum-product algorithm continues to be 
a good approximation to ML decoding. 

The computational complexity of the sum-product algorithm associated with a graphical real- 
ization of a code is largely determined by the sizes of the local constraint codes in the realization. 
In Section [3] of this paper, we define various measures of "constraint complexity" of a graphical 
realization that can be used as estimates of the computational complexity of sum-product decoding. 
These complexity measures may be viewed as generalizations of previously proposed measures of 
trellis complexity lITTl.l 141. and tree complexity [7],[8|. However, for the most part, we focus on 
the ^-complexity of a graphical realization, which we define to be the maximum of the dimensions 
of the local constraint codes in the realization. 

In the restricted context of tree realizations, it has previously been established that certain "min- 
imal" tree realizations can be canonically defined. Let the term tree decomposition denote a graph 
decomposition in which the graph is a tree. It is known that among all tree realizations of a code 
C that extend a given tree decomposition, there is one that minimizes the dimension of the state 
space at each edge of the underlying tree, and this minimal tree realization is unique @. It has 
further been shown [11] that this unique minimal tree realization also minimizes (among all tree 
realizations extending the given tree decompositions) the dimension of the local constraint code at 
each vertex of the tree. In particular, it has the least K-complexity among all such tree realizations. 

In contrast, there is very little known about the general case of realizations of a code on an 
arbitrary (not necessarily cycle-free) graph. For instance, there appear to be no "canonical" minimal 
realizations that can be defined in this situation. The only systematic study in this direction remains 
that of Koetter and Vardy Ifl3l . lfl4"1 . who studied minimal tailbiting trellis representations of codes, 
which, as already mentioned, are graphical realizations in which the underlying graph consists of 
exactly one cycle. Beyond this basic (though by no means easy) case, there is little of interest in 
the literature on the complexity of realizations of codes on arbitrary graphs, the notable exception 
to this being the work of Halford and Chugg (H. 

In their work, Halford and Chugg lay the foundations for a systematic study of complexity of 
graphical realizations. Their main result is the "Forest-Inducing Cut-Set Bound", which gives a 
lower bound on the constraint complexity of a graphical realization in terms of its minimal tree 
complexity. However, this bound does not appear be user-friendly in practice. The main limitation 
of their approach is that they rely on the Edge-Cut Bound of Wiberg et al. liT8l . lTT9l to derive their 
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results. While the Edge-Cut Bound has been put to good use in the study of "state complexity" of 
graphical realizations [5],[6], it is of limited value in the analysis of constraint complexity. 

The main aim of our paper is to present useful and tight lower bounds on the constraint com- 
plexity of a graphical realization. Our bounds also provide considerable insight into the problem of 
finding low-complexity graphical realizations. The fundamental tool in our analysis is the Vertex- 
Cut Bound, which we state and prove in Section [4] The Vertex-Cut Bound is a natural analogue 
of the Edge-Cut Bound, but as we shall see, it is more suitable for use in the analysis of constraint 
complexity. 

In Section[51 we define a data structure called vertex-cut tree that stores the information necessary 
about a graph to effectively apply the Vertex-Cut Bound. Vertex-cut trees are similar in structure to 
the junction trees associated with belief propagation algorithms ifTOl .lfTll. The vc-width of a vertex- 
cut tree is a measure of the size of the vertex-cut tree, and the vc-treewidth of a graph is the least 
vc-width among all its vertex-cut trees. The vc-treewidth of a graph is very closely related to the 
notion of treewidth of graphs much studied in graph theory lfl6l .|[3l. 

Using the Vertex-Cut Bound and the notion of vertex-cut trees, we derive, in Section [6j a suite 
of lower bounds on the K-complexity of graphical realizations of a linear code C. We state one of 
these bounds here as an illustrative example. Let Ktree(C) be the least K-complexity among all tree 
realizations of C. Consider an arbitrary graph Q, and let K V c-tree(<?) denote its vc-treewidth. Then, the 
K-complexity of any realization of C on Q is bounded from below by the ratio K tree (C) / ft vc -tree {G)- 

We further apply our methods to answer certain questions raised in iTTTTl . Borrowing terminology 
from (TTJ, for a code C, let b(C) denote the least edge-complexity of any trellis representation of C 
or any of its coordinate permutations. In the language of our paper, b(C) is the least K-complexity 
of any conventional trellis realization of C. We show that for any linear code of length n, we have 
b[C) / K\xee {C) = 0(log 2 n), and that this is the best possible estimate of the ratio, up to the constant 
implicit in the O-notation. This is used to extend a known lower bound [ 15 ] on b(C) in terms of the 
length n, dimension k and minimum distance d of C, to a lower bound on Ktree(C). 

Our lower bound on K tree (C) has an important implication. It shows that if £ is a code family 
with the property that, for each C € £, fttree(C) is bounded from above by a fixed constant, then 
either the dimension or the minimum distance of the codes in £ grows sub-linearly with codelength. 
Thus, such code families are not good from a coding-theoretic perspective. We also prove a slightly 
more general result, which can be roughly interpreted as saying that a good error-correcting code 
cannot have a low-complexity realization on a graph with small vc-treewidth. So, for good codes, 
if low-complexity graphical realizations exist, then they must necessarily exist on graphs with large 
vc-treewidth. 

Some concluding remarks are made in Section [7J and an example in support of a statement in 
Section[3]is given in an appendix. 

2. Background and Notation 

In this section, we provide the necessary background, and define the notation we use in the 
paper. We take F to be an arbitrary finite field. Given a finite index set /, we have the vector space 
F 1 = {x = (x{ € F, i £ I)}. For x € F 1 and J C I, the notation x| j will denote the projection 
(xi, i € J). Also, for J C J, we will find it convenient to reserve the use of J to denote the set 
{iel: i t J}. 

2.1. Codes. A linear code over F, defined on the index set /, is a subspace C C F 1 . In this paper, 
the terms "code" and "linear code" will be used interchangeably to mean a linear code over an 
arbitrary finite field F, unless explicitly specified otherwise. The dimension, over F, of C will be 
denoted by dim(C). An [n, k] code is a code of length n and dimension k. If, additionally, the code 
has minimum distance d, then the code is an [n, k, d] code. 
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Figure 1. A graph decomposition of I = {1, 2,3, ... , 11}. 

Let J be a subset of the index set I. The projection of C onto J is the code C\j = {c\ j : 
c G C}, which is a subspace of ¥ J . We will use Cj to denote the cross-section of C consisting of 
all projections c|j of codewords c € C that satisfy c| -j = 0. To be precise, Cj = {c\j : c G 
C,c\j = 0}. Note that Cj C C|j. Also, since Cj is isomorphic to the kernel of the projection 
map 7T : C — > C\j defined by ir(c) = c\j, we have that dim(Cj) = dim(C) — dim(C| j). As a 
consequence, we see that if J C K C I, then dim(Cj) < dim(Cj^). 

If C\ and C2 are codes over F defined on mutually disjoint index sets Ii and I 2 , respectively, then 
their direct sum is the code C = C\ © C2 defined on the index set I\ U I2, such that Cj L = C\ j = C\ 
and Ci 2 = C\ l2 = C2. This definition naturally extends to multiple codes (or subspaces) C a , where 
a is a code identifier that takes values in some set A. Again, it must be assumed that the codes C a 
are defined on mutually disjoint index sets I a , a € A. The direct sum in this situation is denoted 
by® aGA C a . 

2.2. Graphs. In this paper, we are primarily interested in graphs that are connected, so any un- 
qualifed use of the term "graph" should be taken to mean "connected graph". Let Q = (V, E) be a 
graph, where V and E denote its vertex and edge sets, respectively. To resolve ambiguity, we will 
sometimes denote the vertex and edge sets of Q by V{Q) and E{Q), respectively. Given a v G V, 
the set of edges incident with v will be denoted by E(v). 

For ICfi,we define Q \ X to be the subgraph of Q obtained by deleting all the edges in X. If 
X consists of a single edge e, then we will write Q\e instead of G \ {e}. If Q \ X is disconnected, 
then X is called an edge cut of Q. Similarly, for W C V, we define Q — W to be the subgraph of 
Q obtained by deleting all the vertices in W along with all incident edges. If W consists of a single 
vertex v, then we will write Q — v instead of G — {v}. If Q — W is disconnected, then W is called 
a vertex cut of Q. 

A tree is a connected graph without cycles. Vertices of degree one in a tree are called leaves, and 
all other vertices are called internal nodes. Note that any subset of the edges of a tree constitutes 
an edge cut, and any subset of internal nodes constitutes a vertex cut of the tree. If e is an edge in a 

tree T, then we will denote by and the two components of T \ e. If v is a vertex of degree 

5 in a tree T, then we will use T x , T 2 , . . . , T) to denote the components of T — v. 

A path is a tree with exactly two leaves (the end-points of the path). All internal nodes in a path 
have degree two. A simple cycle is a connected graph in which all vertices have degree two. An 
n-cycle, n > 3, is a simple cycle with n vertices. 

2.3. Graphical Realizations of Codes. The development in this section is based on the exposition 
of Forney 0,(71; see also [8], [HI. Let I be a finite index set. A graph decomposition of I is a 
pair (£/, uj), where Q = (V, E) is a graph, and u> : I — > V is an index mapping. For a code C, we 
will usually write "graph decomposition of C" as shorthand for "graph decomposition of the index 
set of C". We again wish to emphasize that, unless explicitly stated otherwise, we will take Q to 
be a connected graph. When Q is a tree, (Q, uj) will be called a tree decomposition. Pictorially, a 
graph decomposition (Q, uj) is depicted as a graph with an additional feature: at each vertex v such 
that uj _1 (v) is non-empty, we attach special "half-edges", one for each index in uj^ 1 ^). Figure Q] 
depicts a graph decomposition of I = {1,2, 3,... ,11}. 



CONSTRAINT COMPLEXITY OF GRAPHICAL REALIZATIONS 



5 



For a graph Q = (V, E), recall that E(v), v G V, denotes the set of edges incident with v in Q. 
Consider a tuple of the fornd (Q, u), (S e , e G E), (C v , v G V)), where 

• (£7, lS) is a graph decomposition of /; 

• for each e <E E, S e is a. vector space over F called a state space; 

• for each v G V, C v is a subspace of F w 1 W © (©ee^Ou) <^e) > called a /oca/ constraint 
code, or simply, a /oca/ constraint. 

Such a tuple will be called a graphical model. A graphical model in which the underlying graph Q 
is a tree will be called a tree model. The elements of any state space S e are called states. The index 
sets of the state spaces <S e , e G E, are taken to be mutually disjoint, and are also taken to be disjoint 
from the index set I corresponding to the symbol variables. 

A global configuration of a graphical model as above is an assignment of values to each of the 
symbol and state variables. In other words, it is a vector of the form ((xj G F, i G I), (s e G 
S e , e G E)). A global configuration is said to be valid if it satisfies all the local constraints. 
Thus, ((xj G F, i G I), (s e G <S e , e G S)) is a valid global configuration if for each v G V, 
((xj, i G )), (s e , e G E(v))) G C7„. The set of all valid global configurations of a graphical 

model is called the full behavior of the model. 

Note that the full behavior is a subspace iBCF'ffi (0 ee £<S e ). As usuai > for J Q L %\j 
denotes the projection of 03 onto the index set J. For future convenience, we also define certain 
other projections of 03. Let b = ((xj, i G I), (s e , e G -E)) be a global configuration in 03. At any 
given v E V, the /oca/ configuration of b at u is defined as 

b|„ = ((xi, i G uj- 1 ^)), (s e , e G £(«))). 

The set of all local configurations of 03 at v is then defined as 2$| = {b\ v : b G 03}. By definition, 
%$\ v Q Cv- Similarly, given any e G E, if b is a global configuration as above, then we define 
b| = s e ; we further define Q3| e = {b| e : b G 53}. Clearly, 53| e is a subspace of S e . 

A graphical model (£7,u;, (<S e , e G £?), (C„, v G V)) is defined to be essential if 58^ = C v for 
all f G V and 03 | e = S e for all e £ E. When ^ is a tree, there is some redundancy in the above 
definition, as the condition 03 | e = S e for all e G E actually implies that 58 | u = C„ for all v G V 
iTTTl Lemma 2.2]. It is worth noting that, in an essential graphical model, at any edge e = {u, v}, 
the state space S e , may be viewed as a projection of each of the local constraint codes C u and C v . 
Thus, for any e G E, dim(5 e ) < dim(C7^), where v is any vertex incident with e. 

An arbitrary graphical model V with full behavior 03 can always be "essentialized" by simply 
replacing each local constraint C v in V with the projection *B\ V , and replacing each state space S e 
with the projection Q3| e . The resulting "essentialization" of F still has full behavior 03. 

An essential graphical model (Q,u, (S e , e G E), (C v , v G V)) is denned to be a graphical 
realization of a code C, or simply a realization of C on Q, if 03 1 7 = C. A graphical realization of 
C in which the underlying graph Q is a tree, is called a ?rcc realization of C. Our definition of a 
graphical (and tree) realization differs slightly from the prior definitions in [61,lf71.ll8l. lfTTi . in that 
we require the underlying graphical model to be essential. As explained above, any graph model 
can be essentialized, so there is no loss of generality in this definition. 

A graphical (resp. tree) realization (Q,u>, (<S e , e G E), (C v , v G V)) of C is said to extend, or 
be an extension of, the graph (resp. tree) decomposition (Q, u>) of C. We will denote by 9t(C; G, oj) 
the set of all graphical realizations of C that extend the graph decomposition (Q, ui) of C. 

Now, it is an easily verifiable fact that any tree decomposition of a code can always be extended 
to a tree realization of the code lITl. lfTTll . Such an extension is not unique in general, but we will 
describe a canonical "minimal" extension a little later. More generally, any graph decomposition 
of a code C can always be extended to a graphical realization of C, as we now explain. Let Q = 



In referring to a tuple of the form (G,uj, (5 e , e £ E), (C v , v G V)), we will implicitly assume that V and E denote 
the vertex and edge sets, respectively, of the graph Q. 
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(V, E) be a connected graph, and suppose that (Q,lo) is a graph decomposition of C. Take T to 
be any spanning tree of Q, and let Et denote its edge set. Then, (T,u>) is a tree decomposition of 
C. As noted above, this tree decomposition can be extended to a tree realization (T, u, (S e , e G 
Et), (C v , v G V)) of C. We further extend this to a realization of C on Q as follows. Define the 
state spaces S e , e G E, as 

- = (S e ifeeE T 
e {{0} if e£E\E T , 

and for each v G V, define the local constraint C v = C v © ((B e £E(v)\E T {®})- ^ should be clear 
that (G,ui, (S e , e G E), (C v , v G V)) is a graphical realization of C. 

Graphical realizations of codes in which the underlying graph is a path or a simple cycle have 
received considerable prior attention in the literature. Such realizations were called "conventional 
state realizations" (when the underlying graph is a path) and "tail-biting state realizations" (when 
the underlying graph is a simple cycle) in 0. We will call them "trellis realizations". Briefly, a 
trellis realization of a code C (defined on the index set /) is any extension of a graph decomposition 
of C of the form (Q, u>), where Q is either a path or a simple cycle, and u is a surjective map lu : I —> 
V(Q). A trellis realization in which the surjective map uj : I —>■ V(Q) is not injective (so that uj is 
not a bijection), is usually called a sectionalized trellis realization. A conventional trellis realization 
is one in which the underlying graph Q is a path. When the underlying graph Q is a simple cycle, 
the trellis realization is said to be tailbiting. The theory of conventional trellis realizations is well 
established; see, for example, [17]. On the other hand, tailbiting trellis realizations are less well 
understood; the principal systematic study of these remains that of Koetter and Vardy lfT3l . lfl4l . 
We remark that our requirement that graphical realizations have underlying graph models that are 
essential corresponds to the requirement in lfT3l . lfl4l that "linear trellises" be "reduced". 



3. Complexity Measures for Graphical Realizations 

As observed in [6 ], any graphical realization of a code specifies a class of associated graph-based 
decoding algorithms, namely, the sum-product algorithm and its variants. Thus, ideally, any defi- 
nition of a complexity measure for a graphical realization should try to capture the computational 
complexity of the associated decoding algorithms. The analysis in (6., Section V] shows that the 
computational complexity of the sum-product algorithm specified by a given graphical realization 
of a code is determined in large part by the cardinalities, or equivalently dimensions, of the local 
constraint codes in the realization. Thus, as a simple measure of the complexity of a graphical 
realization, which roughly reflects the complexity of sum-product decoding, we will consider the 
maximum of the dimensions of the local constraint codes in the realization. 

Let T = (Q, u, (C v ,v G V), (<S e , e G E)) be a graphical realization of a code C. The constraint 
max-complexity , or simply n-complexity, of T is defined to be n(T) = max^gy dim(C„). Now, 
recall that if (Q, uj) is a graph decomposition of C, then 9\(C; Q, uj) denotes the set of all graphical 
realizations of C that extend (Q, uj). We further define 

h(C;G,uj) = min k(T) (1) 

rem(C;S,w) 

We add another level of minimization by defining, for a given graph Q, the Q-width of a code C to 
be 

k(C;G) = min k(C;G,uj), (2) 

LU 

where the minimum is taken over all possible index mappings uj : I — > V(Q), where / is the index 
set of C. Thus, the (y-width of C is the least ft-complexity of any realization of C on Q, and may 
be taken to be a measure of the least computational complexity of any sum-product-type decoding 
algorithm for C implemented on the graph Q. 
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A broader optimization problem of considerable interest is the following: given a code C and 
a family of graphs (25, identify a Q G (J5 on which C can be realized with the least possible re- 
complexity. We thus define 

k(C;<&) = mm K(C;g). (3) 
gee 

Two special cases of this definition — treewidth and pathwidth (or trellis-width) — are particularly 
of interest. We define treewidth first, and pathwidth a little further below. 

If we let T denote the set of all trees, then k(C; X) is called the treewidth of the code C ifTTTl . 
which we will denote by K tree (C). The notion of treewidth {i.e., minimal K-complexity among tree 
realizations) of a code was first considered by Forney [7 ], and an analogous notion has been defined 
for matroids in [9]. The arguments in [7, Section V] (and also in [9]) show that K tree (C) can always 
be obtained by minimizing k(C; T, lo) over tree decompositions (T, lo) in which T is a cubic tree 
(i.e., a tree in which all internal nodes have degree 3), and to is a bijection between the index set of 
C and the set of leaves of T. 

A complexity measure related to treewidth, termed minimal tree complexity, was defined and 
studied by Halford and Chugg [8]. Treewidth, as we have defined above, is an upper bound on the 
minimal tree complexity of Halford and Chugg. 

The pathwidth, K pat h(C), of a code C is defined to be the quantity k(C; ty), where *p denotes the 
sub-family of T consisting of all paths. We will find it convenient to refer to tree decompositions 
(P,lo), with P G ty, as path decompositions. Thus, k(C;^) is the minimum value of k(C;P,lo) 
as (P, lo) ranges over all path decompositions of C. In fact, by the argument of [7, Section V.B], 
the minimizing path decomposition (P, a;) may be taken to be one in which the index mapping lo is 
surjective. Thus, K pa th(C) is the least ^-complexity of any conventional trellis realization of C, and 
so we may also call it the (conventional) trellis-width^ of C. It is also known that sectionalization 
cannot reduce the K-complexit)0 of a trellis realization iMTl Theorem 6.3], and hence, K pat h(C) is 
the minimum value of k(C; P, lo) over all path decompositions (P, lo) in which the index mapping 
lo is a bijection between the index set of C and the vertices of P. 

Measures of constraint complexity other than K-complexity have been proposed in the previous 
literature, especially in the context of trellis realizations lfI71 . lfl4ll . The k + -complexity of a graph- 
ical realization T = (G,lo, (C v , v G V), (S e , e G E)) is defined to be k + (T) = X^vev dim(C„). 
Note that jyj- n + (T) is the average local constraint code dimension in T. On the other hand, it has 
been suggested [7 ] that the sum of the constraint code cardinalities "may be a better guide to de- 
coding complexity" than ^-complexity or k + -complexity. Thus, we define K tot (r) = YlveV \C V \ = 
T,vev \¥\ dim ^. Analogous to ©-©, we may define k + (C,Q,lo), K tot (C;G,Lo), etc., but we will 
only touch upon these briefly in this paper. 

We remark that while we have used constraint code dimensions to define our complexity mea- 
sures for graphical realizations, one could also define measures of complexity based on state-space 
dimensions. For example, we could define the state max-complexity, <r(r), of a graphical realiza- 
tion T to be the maximum of the dimensions of the state spaces in T. Similarly, we may consider the 
complexity measures c + (r) and a tot (r) analogous to k + (T) and K tot (r). These measures are espe- 
cially relevant and have been well studied in the context of trellis realizations; again, see Ifl7ll . lfl4ll . 
However, as noted by Forney Q, measures of state-space complexity become less appropriate in 
the context of realizations on arbitrary graphs or trees. For example, for any code C C F n , one can 
always find a tree on which C can be realized in such a way that all state-spaces have dimension at 
most 1. This would be the "star-shaped" tree T consisting of n leaves connected to a single internal 
node v of degree n. Take lo to be any bijection between the index set of C and the leaves of T; set 
S e = F at each edge e of T; and finally, take the local constraint code C v at the internal node to be 
C itself, and take the local constraint codes at the leaves to be [2, 1] repetition codes. Clearly, the 



For this reason, what we have called K pa ih(C) here was called Ktreiiis(C) in II II . 
'Our notion of K-complexity corresponds to the notion of "edge-complexity" in 1171 . 
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resulting tree model (after essentialization) is a tree realization of C. Thus, it makes little sense to 
define a state-space complexity measure analogous to treewidth, unless we restrict the kind of trees 
on which we are allowed to realize the given code0. 

The astute reader may point out that in the trivial tree realization above, the sum of the state- 
space dimensions is non-trivial, and so a state-space analogue to treewidth could potentially be 
defined in terms of a + or cr tot . This may be true, but we do not pursue this further, since, as already 
observed previously, complexity measures based on constraint code dimensions are a better guide 
to decoding complexity. But, while on this topic, we mention in passing that for any tree realization 
T of a code C, it turns out that 



Thus, the problem of minimizing cr + (T) among tree realizations T of a given code C is equivalent 
to the problem of minimizing k + (T). The identity in (01), which may be viewed as a generalization 
of the statement of Theorem 4.6 in |[T4l for conventional trellis realizations, will not be proved here 
as it would be an unnecessary deviation from the main line of our development. It suffices to say 
that dU) follows from Theorem 3.4 in ifTTI by first verifying that it indeed holds for any minimal 
tree realization M(C;T,ui), and then observing that the difference between k + (T) and <t + (T) is 
preserved by the state-merging process mentioned in the statement of that theorem. 

Finally, we remark that the state max-complexity of a graphical realization cannot exceed the 
constraint max-complexity of the realization. This is because, as observed in Section 12.31 in 
any essential graphical model (G,uj, (<S e ,e G E), (C v ,v G V)), for each edge e G E, we have 
dim(iS e ) < dim(Cy), where v is any vertex incident with e. 

3.1. Minimal Realizations. Given a code C and a tree decomposition (T,uj) of C, there exists a 
tree realization, (T,u, (S*, e G E), (C*, v G V)), of C with the following property 10,0: 

if (T,u>, (<5 e , e G E), (C v , v G V)) is a tree realization of C that extends (T,ui), 
then for all e G E, dim(«S*) < dim(«S e ). 

This minimal tree realization, which we henceforth denote by M.{C; T, uj), is unique up to isomor- 
phism!]. Constructions of M(C;T,u>) can be found in |[6l0.lfTT1. 

It has further been shown iTTTTl that not only does M(C; T, uj) minimize (among realizations in 
9t(C; T, uj)) the state space dimension at each edge of T, but it also minimizes the local constraint 
code dimension at each vertex of T. More precisely, M.(C; T, uj) also has the following property: 

if (T,u>, (S e , e G E), (C v , v G V)) is a tree realization of C that extends (T,u>), 
then for all v G V, dim(C*) < dim(C„). 

Consequently, we have that k(C;T,u) = k(M{C;T,u)), k + (C;T,uj) = k + (M(C;T,uj)), and 
K tot (C;T,uj) = K tot (M(C;T,uj)). The fact that M(C;T,u) minimizes local constraint code di- 
mension at each vertex of T will be central to the derivation of our results in the sections to follow. 

We will henceforth consistently use the notation S* and C* to denote state spaces and local 
constraint codes in a minimal tree realization A4(C;T,u). Exact expressions for the dimensions 
of S* and C* in M(C;T,lu) are known 0,0. Recall that for an edge e of T, we denote by 

and T (e) the two components of T \ e. Let us further define J(e) = u' x {V{T^)) and 

J( e ) = u~ l (y(^ {e) )). We then have 



We do get a reasonable state-space analogue to treewidth if we restrict the class of trees over which we attempt to 
minimize state-space complexity to the class of cubic trees only; see 1111 . 

5 Graphical realizations (G,w, (C v , v £ V),(S e ,e £ E)) and (Q, lo, (C' v , v G V),(S' e ,e G E)) of a code C are said 
to be isomorphic if, for each v £ V, C v and C' v are isomorphic as vector spaces, and for each e 6 E, <S e and S' e are 
isomorphic as vector spaces. We do not distinguish between isomorphic graphical realizations. 



/c+(r) = dim(C) + cj+(r). 



(4) 



dim(<S*) = dim(C) — dim(C/( e )) — dim(Cj^x). 



(5) 
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Next, consider any vertex v in T. If v has degree 5, then T — v has components T- , i = 1, 2, . . . , 6. 
Define J; = uj~ l {V {T^ )) , for i = 1, 2, . . . , 8. Then Theorem 1], 

dim(C: ) = dim(C) - J2 dim ( C Ji ) • ( 6 ) 

i=l 

In summary, the minimal tree realization Ai (C; T, uj) is an exact solution to the problem of deter- 
mining the minimum-complexity extension of a tree decomposition (T, uj) of a code C. Moreover, 
M.{C; T,lu) minimizes, among realizations in 1K(C; T, uj), any reasonable measure of complexity, 
be it state-space complexity or constraint complexity. Unfortunately, when we move to realizations 
on graphs with cycles, there appear to be no "canonical" minimal realizations with properties sim- 
ilar to those of minimal tree realizations. In fact, if {Q,ui) is a graph decomposition of a code C, 
where Q is a graph with cycles, there need not even be a realization T that simultaneously achieves 
mm re <K(C;g, w ) k(T) and min rg f^( C .g iU \ k + (T). An example of such a graph decomposition is given 
in Appendix El 

Thus, given a code C and a graph Q containing cycles, the problem of finding realizations of C on 
Q with the least possible K-complexity, k + -complexity, or K tot -complexity (within some interesting 
sub-class of realizations of C on Q) is much harder to solve than the corresponding problem for 
cycle-free graphs. In the next section, we present a simple but valuable tool that will enable us to 
derive non-trivial lower bounds on the constraint complexity of realizations of a code on an arbitrary 
graph. These bounds could be used, for example, to determine whether or not the complexity of a 
given realization is close to the least possible. 

4. Cut-Set Bounds 

Let Q = (V, E) be a connected graph. A partition (V, V") of V is said to be separated by an 
edge cut X C E if , for each pair of vertices v' G V and v" G V", any path in Q that joins v' to v" 
passes through some edge e G X. We remark that if Q \ X has more than two components, then 
there is more than one partition of V that is separated by X. 

The Edge-Cut Bound, stated below, is a result of fundamental importance in the study of graphi- 
cal realizations. This result was originally observed by Wiberg, Loeliger and Koetter Ifl8ll , |fl9ll , but 
the version we give here is due to Forney [6, Corollary 4.4]. 

Theorem 4.1 (The Edge-Cut Bound). Let V = (Q, uj, (C v ,v € V), (<S e , e G E)) be a realization of 
a code C on a connected graph Q = (V, E). If (V' , V") is a partition ofV separated by an edge 
cut X C E, then, defining J' = ui~ l {y') and J" = u)~ l (V"), we have 

dim(5 e ) > dim(C) — dim(Cj/) — dim(Cj»). 

The edge-cut bound can be used to derive useful lower bounds on the state-space complexity of 
a graphical realization; see, for example, Q. To deal with constraint complexity, however, we will 
need a closely-related bound that uses vertex cuts instead of edge cuts. 

We introduce here some terminology that we will use to state our vertex-cut bound. For v G V, 
let N(v) = {u G V : {u, v} G E} denote the set of neighbours of v in Q. Furthermore, for 
WQV, define N{W) = \J v€W N(v). 

Definition 4.1. An ordered collection (Vb, V{, . . . , Vg), 5 > 0, of subsets of V is said to be a star 
partition ofV, if the Vi's form a partition ofV (i.e., the Vi's are pairwise disjoint, and Ui=o ^ = 
V), and for each i G {1, 2, . . . , 6}, we have N(Vi) QViU V . 

The definition has been worded so as to allow some of the Vi's to be empty sets. When V{ is 
non-empty for at most one i > 1, a star partition is simply a partition. When at least two V^'s 
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other than Vq are non-empty, then a star partition is a partition that arises from a vertex cut of Q, 
as we now explain. For any i > j > 1, if Vi and Vj are both non-empty, then the above definition 
simply says that any path between a vertex in Vi and a vertex in Vj must pass through Vq. Thus, if 
at least two V^'s other than Vq are non-empty, then Vq is a vertex cut of Q. Conversely, if Vq is a 
vertex cut of Q, and Q\, Q2, ■ ■ ■ , Qs, 5 > 2, are the (non-empty) components of Q — Vq, then, setting 
Vi = V(Gi) for i = 1, 2, . . . , S, we see that (Vq, Vi,. . . , V$) is a star partition of V. The graph on 
the left in Figure [H which depicts a typical situation covered by the definition, should also explain 
the nomenclature. 

Theorem 4.2 (The Vertex-Cut Bound). Let T = (Q, u, (C v ,v G V), (S e , e G E)) be a realization 
of a code C on a connected graph Q = (V, E). If {Vq, V%, . . . , V$) is a star partition of V, then, 
defining Ji = uj^ 1 (Vj) for i = 1,2, ... ,5, we have 

s 

dim(C v ) > dim(C) - ^dim(CjJ. 



Proof. Let *B denote the full behaviour of T. If b = ((x{,i G I) , (s e , e G E) ) is a global configura- 
tion in 03, then given an X C E, we will use h\ x to denote the projection (s e , e G X). We further 
set = {h\ x : b G 03}. 

For i = 1, 2, let Xj be the set of edges of £ with exactly one end-point in Vi, so that the 

other end-point is necessarily in Vq. We then define 

2% = {( b U' b U) : b£«8}, 
for i = 1, 2, . . . , S. Furthermore, set Jo = u>~ 1 (Vq) and Xq = Ui=i an d define 

«lv = {( b L' b k) : bG<B >- 

Now, consider the "star-shaped" tree T = (Vy, £t) consisting of a single internal vertex vq of 
degree 5, whose neighbours ui,t>2, • • • , vg are all the leaves of T. Thus, Vr = {^>o> vi, . . . , 1^,5} and 
Et = {{vq, Vi} : i = 1,2, ... ,5}. Define the index mapping a : I — ► Vr as follows: a(j) = Vi iff 
w(j) G V^. Note that, for i = 0, 1, 2, ... , <5, we have a _1 (Fi) = cj _1 (Vi) = Jj. The construction 
of the tree decomposition (T, a) from (Q, ui) is depicted in Figure[2l 
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We next extend the tree decomposition (T,a) to a tree model T = ((T,a, (<S e ,e G Er), (C v ,v G 
Vr)) by setting C v . = 58|y. for i = 0, 1, 2, ... , 5, and S^,^} = 58| x . for i = 1,2, ... ,5. From 

the fact that T is a realization of C, it readily follows that T is a tree realization of C. 

Recalling that the minimal tree realization M.(C; T, a) minimizes the local constraint code di- 
mension at each vertex of T, we obtain via ©, 

5 

dim(C% ) >dim(C)- j^dimCCjJ. 

i=i 

We complete the proof by observing that 

dim(C% ) = dim(58y < ]T dim(58|J = £ dim(C7 v ). 

□ 

The following useful corollary is an immediate consequence of the Vertex-Cut Bound. 

Corollary 4.3. Let (Q,uj) be a graph decomposition of a code C, where Q is a connected graph. 
For a vertex cut W of Q, if Gi,G%, ■ ■ ■ ,Gs are the components of Q — W, then define X(W) = 
dim(C) — Y2i=i di m (CjJ, where J{ = (V (Gi)) for i = 1,2, ... ,5. Then, for any realization 
T = (Q, to, (C v ,v £ V), (S e , e G E)) of C that extends (Q, lu), we have 

dim(C v ) > \{W). 

In its most straightforward application, the Vertex-Cut Bound, via the above corollary, can be 
used in conjunction with constrained optimization techniques to find lower bounds on k(C; G, u>), 
k + (C; G, oj) and n tot (C; G, w). Indeed, if (G, u>) is a graph decomposition of C, and W\, W2, ... ,Wt 
are vertex cuts of G, then, by Corollary 14.31 the dimensions of the local constraint codes C v , v G V, 
in any T G 5H(C; G, must satisfy Y^v&Wi di*n-(C v ) > X(Wi), i = 1, 2, . . . , t. Thus, for example, 
k + (C; G, oj) is lower bounded by the solution to the following linear programming problem in the 
variables £ v , v G V: given a collection of vertex cuts W±, W2, ■ ■ ■ , Wt of G, 

minimize £„, subject to £ w > \{Wi), i = 1, 2, . . . , t. 

However, we do not pursue this angle any further in this paper. Instead, we will henceforth re- 
strict our attention to the K-complexity measure, for which we will derive a suite of lower bounds, 
again based on the Vertex-Cut Bound, which unearth some interesting connections with graph the- 
ory, and moreover, are amenable to further mathematical analysis. The bounds we derive rely on 
the notion of vertex-cut trees introduced in the next section. 

5. Vertex-Cut Trees 

We begin with a simple lemma, which plays a role in our definition of a vertex-cut tree below. 

Lemma 5.1. Suppose that Vq, V\, . . . , V$ are subsets ofV such that (Ji=o ^» = V- V>f or eac ^ P a ^ r 
of distinct indices i,j, we have V{ fl Vj C Vq, then {Vq, V\ \ Vq, . . . , V$ \ Vq) is a partition ofV. 

Proof. It is evident that V U Ui=i(^ \ ^0) = ULo V i = V - If »,i > 0, i ^ j, then (Vi \ V ) n 
(Vj \ Vq) = {Vi n Vj) \ Vq = 0, since V n Vj C V . □ 

For the main definition of this section, we introduce some convenient notation, which will hence- 
forth be used consistently. If A and B are sets, and / : A — > 2 B is a mapping from A to the power 

set of B, then for any X C A, we define f(X) = IJ^gx f( x )- Also, recall that if z is a vertex of 

(z) 

degree 5 in a tree T, then the components of T — z are denoted by T- , i = 1, 2, . . . , 0. 
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Definition 5.1. Let Q be a connected graph. A vertex-cut tree ofQ is a data structure (T, 0), where 
T is a tree, and : V(T) — » 2 V ^> is a mapping with the following properties: 
(VC1) P(V(T)) = V@); 

(VC2) for each pair x, y G V(T), if z G V(T) is any vertex that lies on the unique path between 

x and y in T, then 0(x) fl 0(y) C /3(z); 
(VC3) /or eac/j 2 G V"(T), (f3(z), V\, V2, ■ ■ ■ , V£) w a star partition ofV(Q), where 5 is the degree 

ofz, and Vi = (3{V(T t {z) )) \ 0(z)for i = 1, 2, . . . , 5. 
A vertex-cut tree (T, 0) in which T is a path is called a vertex-cut path. 

Note that, by Lemma [5TT1 conditions (VC1) and (VC2) in the above definition imply that, for 
each z G V(T), with 5 and Vi, i = 1,2,... ,5, as in condition (VC3), {(3{z), Vi,V 2 ,..., Vg) is a 
partition of V(Q). Thus, for (T, 0) satisfying conditions (VC1) and (VC2), condition (VC3) is met 
iff, for each z G V(T), we have N{V{) C Vi U 0{z) for all i > 1. 

Trivial vertex-cut trees (and paths) always exist for a graph Q — given any tree T, pick a vertex 
zq G V"(T), and define a mapping /3 by setting /3(zq) = V{Q), and 0{z) = for z 7^ zo- This 
allows us to make the following definition. 

Definition 5.2. Let Q be a connected graph. The vc-width of a vertex-cut tree (T, 0) ofQ is defined 
as max 2g y( T ) and is denoted by vc-width(T, 0). The vc-treewidth (resp. vc-pathwidthj of 

Q is the least vc-width among all vertex-cut trees (resp. vertex-cut paths) of Q, and is denoted by 

^vc-tree 

(Q) (resp. 

^vc-path 

Thus, for any graph Q, we have < K vc -tree(^) < ^vc-path(^) < |V(£7)|. The vc-treewidth of any 
tree is equal to one. Indeed, if T is a tree, then (T, 0), defined by 0(z) = {z} for all z G V(T), is 
a vertex-cut tree of T, with vc-width equal to one. 

Example 5.1. Let Q be an n-cycle with vertices v 0, V\ , . . . , v n -\, labeled in cyclic order. Let P be a 
path with n—1 vertices, which in the linear order defined by the path, are labeled Z\, Z2, ■ ■ ■ , Z n -\. 
To be precise, zq is one of the two leaves, and for % = 1, 2, . . . , n — 1, z\ is adjacent to z%—\ in P. If 
we define the mapping (3 : V(P) — > 2 V ^ as 0(zi) = {vq, v^} for i = 1, 2, . . . , n — 1, then (P, 0) 
is a vertex-cut path ofQ, of vc-width two. It is not difficult to verify that Q has no vertex-cut tree of 
vc-width one, and hence, K vc -tree(^) = ^vc-path(^) = 2. 

Our definition of vertex-cut trees may appear at first to be an artificial construct brought in solely 
for the purpose of finding applications for the Vertex-Cut Bound. However, this is far from being 
the case. Vertex-cut trees are very closely related to junction trees, commonly associated with belief 
propagation algorithms in Bayesian networks [10] and coding theory (TJ. Another close relative of 
vertex-cut trees is the data structure known as tree decomposition (of a graph) [16],[2],[3], which 
has received considerable attention in the graph theory and computer science literatures. 

Definition 5.3. A tree decomposition of a connected graph Q is a data structure (T, 0), where T is 
a tree, and : V(T) — * 2 V ^ is a mapping with the following properties: 
(Tl) 0{V(T)) = V(G); 

(T2) for each pair x, y G V(T), ifzE V(T) is any vertex that lies on the unique path between 

x and y in T, then 0(x) fl 0(y) C 0(z); 
(T3) for each pair of adjacent vertices u, v G V(Q), there exists a z G V(T) such that {u, v} C 

0(z). 

A tree decomposition (T, 0) in which T is a path is called a path decomposition. 

The width of a tree decomposition as above is defined to be max z&v ^ \0(z) \ — 1. The treewidth 
(resp. pathwidth) of a graph Q, denoted by K tvee (G) (resp. 

K path (£?))> is the minimum among the 
widths of all its tree (resp. path) decompositions. Note that if Q has at least one non-loop edge, 
then, because of (T3), any tree decomposition of Q must have width at least one. Thus, for any 
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(b) (c) 

Figure 3. (a) A graph Q on eight vertices, (b) A tree decomposition of Q with 
width 2, and vc-width 3. (c) A vertex-cut path of Q of vc-width 3 that is not a path 
decomposition of Q. 



such graph Q, we have < K tree (Q) < ^path(^) < 1^(^)1 ~~ 1- It is not hard to check that 
any tree with at least two vertices has treewidth equal to one0, and that if Q is an ra-cycle, then 

«tree(£) = K path (Q) = 2. 

Lemma 5.2. Any tree decomposition of a connected graph Q is also a vertex-cut tree ofQ. Hence, 

Kvc-tree(^) < Ktree(£) + 1, and K vc -path(£) < K path (^) + 1. 

Proof. Let (T, (3) be a tree decomposition of Q. We only have to show that (T, (5) satisfies condition 
(VC3) of Definition l5.ll Consider any z E V(T), with degree 5, and let Vi, i = 1, . . . , S, be defined 
as in condition (VC3). By virtue of (Tl), (T2) and Lemma IBTTl (f3(z), V\, . . . , Vg) is a partition of 
V(Q). Thus, we must show that N(Vi) C Vi U (3{z) for i = 1, . . . , 6. 

Suppose, to the contrary, that there exists a v E V. suc h that for some u € N(v), we have 
u £ Vi U Thus, by definition of V u we have u £ f3{V{T^ z) )) U while v G \ (3{z) 

(z) 

for some x G V(IJ ). Now, by condition (T3), we have {u, v} C for some y G V(T). In 
particular, v G /5(x) n On the other hand, by our assumption on u, we must have y ^ z and 

(z) (z) 

y f V(T^ ). Thus, y G V(T- ) for some j 7^ i This means that the vertex z lies on the unique 
path in T joining x and y, and hence, by (T2), v G @(z), which contradicts the existence of v as 
postulated. □ 

A graph Q can have a vertex-cut tree that is not a tree decomposition. Figure [3] shows an exam- 
ple of a vertex-cut path of a graph Q that is not a path decomposition of Q. Also, the inequality 
^vc-tree(^) < ^tree(^) + 1 in Lemma l5T2l can hold with equality — for the graph Q in Figure[3la), it 
is possible to show that K tree (£?) = 2, while K V c-tree(^) = ^vc-path(^) = 3. 

6. Lower Bounds on k-Complexity 

Vertex-cut trees allow us to derive lower bounds on the K-complexity of a graphical realization 
of a code, as we now show. Let (Q, lo) be a graph decomposition of a code C, and let (T, (5) be a 



Without the ' — 1' in the definition of width of a tree decomposition, a tree with at least two vertices would have treewidth 
equal to two. 
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FIGURE 4. The mapping a : V(T) — > 2 V{ - G \ with the vertex z* chosen as shown, 
constructed from the vertex-cut tree in Figure [3jb). 



vertex-cut tree of Q. For each z G V(T), set Jj = lo 1 (Vi) for i = 1, 2, . . . , S, where the Vj's are 
as defined in condition (VC3) of Definition l5.il Further define 

5 

m{z) = dim(C) — ^^dim(CjJ. (7) 

i=l 

Now, consider any graphical realization, T = (Q, lo, (S e , e G E), (C v , v G V)), of C that extends 
(G,tjo). By the Vertex-Cut Bound, we have, for each z G V{T), J2ve/3(z) dim(C v ) > m(z). Since 
^vep(z) dim(C„) < (max v&v ^ g) dim(C v )) \(3(z)\ = k(T) \@(z)\, we obtain 

K(T)\(3(z)\>m(z). 

Maximizing over all z G V(T), we get 

«(T) • vc-width(T, /?) > max m(z) = /x(C; w, /3) (8) 

2ey(T) 

We thus have the following proposition. 

Proposition 6.1. Let (Q, to) be a graph decomposition of a code C, and let (T, 0) be a vertex-cut 
tree ofQ. Then, for any graphical realization V G 9t(C; Q, to), we have 

M(C; W ,/3) 



vc- width (T, (3) 



The lower bound in the above proposition can be brought into a form more convenient for further 
analysis. To do this, we will construct a tree decomposition (T, 7) of C such that p,(C; lo, (5) = 
k(C;T, 7). So, once again, let (G,to) be a given graph decomposition of C, and (T,f3) a given 
vertex-cut tree of Q. We will first give a recipe for constructing tree decompositions of C from 
(Q, lo) and (T, j3). Then, we will show how the main ingredient of the recipe may be chosen so that 
the resulting tree decomposition (T, 7) satisfies p,(C; lo, (5) = k(C; T, 7). 

For any pair of vertices x,y G V(T), let (x,y] denote the set of vertices on the unique path 
between x and y in T, including y, but not including x. Also, we will use f3(x, y], instead of the 
more cumbersome (3((x, y]), to denote the set U z e(z y ] P( z )- Pi°k an arbitrary vertex z* in V(T). 
Define the mapping a : V(T) -> 2 V ^ as follows: a(z*) = @{z*), and for z ^ z*, 

a(z) = f3(z)\(3(z,z*}. 

An example of such a mapping a constructed from the vertex-cut tree (T, (3) in Figure [3^b) is 
depicted in Figure [4] 

We record in the next two lemmas some properties of the mapping a that we use in the sequel. 
Recall our convention that for X C V(T), a(X) = [J xeX a(x). 
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FIGURE 5. A depiction of the situation when z* ^ V(T^). Dashed ovals repre- 
sent components of T — z. 

Lemma 6.2. For z € V(T), ifT^ is any component ofT — z, then 

a(V(TM)) = {^ Vi ? {Z) » ifz * G V ^ 

K V " \/3(y(T( 2 )))\/3(z) ifz*£V(TW). 

Proof. Suppose first that z* G V(T^), and set X = V(T^) \ {z*}. It then follows from the 
definition of a that 



1J a(x) = ( [j (3(x)) \{3(z*)- 



Therefore, a(V(T^)) = a{X) U a(z*) = {(3{X) \ (3{z*)) U (5{z*) = (i{V{T^)). 

Now, consider the case when z* ^ V{T^ Z '\ as depicted in Figure[5] In this case, the definition 
of a implies that 

U «(*)=( U 0(x))\P[z,z*], 

where [z, z*] denotes the set of vertices on the path between z and z* in T, including both z and z*. 
Note that, as a consequence of condition (VC2) of Definition l5.ll we have, for any x £ V(T^ Z >), 
(3{x) n 0[z, z*] = f3{x) n (5{z). Hence, 

U Pi?) ) \ P[z, z*} = ( U fa))\Mz), 

v zeV(TM) / \xeV(T( z )) / 

which proves the desired result. □ 

Lemma 6.3. The sets a{z), z £ V(T), form a partition ofV{Q). 

Proof. Let t\ z , i = 1, 2, . . . , 8, denote the components of T — z* . Observe that, by Lemma I6T21 
(J a(z) = [ja(V{TP)) = U^(f'))\^) = ( IJ 

2^2* j = l j = l \2^2* / 

Therefore, UzgV(T) a ( z ) = L^ev^T) P( z ) = ^(^)> condition (VC1) of Definition l5.il 

Next, we want to show that a(z) n a(z') = for z ^ z'. This is true by definition of a if either 

z = z* or z' = z*. So, we henceforth assume that z and z' are distinct vertices in V(T) \ {z*}. 
Suppose that there exists au£ a(z) n a(z'). We then have 
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(1) v G (3[z), but v £ (3{y) for any y G (z, z*\, and 

(2) v G (3{z r ), but w ^ (3{y) for any y G (Y, 2*]. 

In particular, we see that v ^ /3(-?*). It also follows from (1) and (2) that z' ^ [z,z*], and z ^ 
(z', z*}, which together imply that z* lies on the path between z and z'. However, by (VC2), this 
means that v G /3(z*), a contradiction. □ 

We now construct a tree decomposition (T, 7) of the index set, /, of C, by defining an index 
mapping 7 : / — > V{T) as follows: for each i G I, set 7(2) = zif uj(i) G a(z). In other words, for 
each z G V(T), 7~ 1 (z) = u)~ l (a(z)). Since the sets z G V(T), form a partition of V(Q), 
the mapping 7 is well-defined. 

Example 6.1. Suppose that C is a code of length 10, defined on the index set I = {1, 2, . . . , 10}. 

For the graph Q shown in Figure^a), consider the graph decomposition (G,uj) of C defined by 
w(l) = w(2) = A, w(3) = w(4) = w(5) = B, w(6) = w(7) = E, w(8) = F, W w(9) = 
cj(10) = -ff. Le? (T,P) be the vertex-cut tree of Q in Figure\3\b), from which we obtain the 
mapping a : V(T) — » 2 l/ ^ depicted in Figure^ Based on the last figure, we will use the labels 
z*, A, D, G, F and H to identify the vertices of the tree T. Then, the index mapping 7 : i" — > V{T) 
is given by 7(1) = 7(2) = A, 7(3) = 7(4) = 7(5) = 7(6) = 7(7) = z*, 7(8) = F, and 
7(9) = 7(10) = H. 

Note that our construction of the tree decomposition (T, 7) depends on the choice of the vertex 
z* , via the mapping a. Up to this point, our choice of z* was arbitrary. We now specify z* G V(T) 
to be such that m{z*) = p,(C; co, (5). We claim that for the tree decomposition (T, 7) arising from 
such a choice of z*, we have p,(C; to, (5) = k(C; T, 7). 

Proposition 6.4. Given a graph decomposition (Q, u) of a code C, and a vertex-cut tree (T, (3) of 
Q, there exists a tree decomposition (T, 7) ofC such that p,(C; u, (3) = k(C; T, 7). 

Proof. Pick a z* G V(T) such that m(z*) = /j,(C;uj, (3), and construct the tree decomposition 
(T, 7) as described above. Note that, by ©, 

K(C;T,i) = k(M(C;T,j)) = max k(z), 

z&V(T) 

where, for a vertex z G V(T) of degree S, k(z) = f dim(C) — J^^ =1 dim(Ci^- i ), with Ki = 
7-1 (v{T^ z) )^ for i = 1,2, ... ,8. Recall from the definition of 7 that ^(z) = o;- 1 (a(z)) 

for any z G V(T). Hence, iv", = lo' 1 (a(V (T t (z) '))) , j = 1, 2, . . . , 5. 

To prove the proposition, we show that k(z) < m(z) for all z G V(T), with equality holding 
if z = z*. From ©, we see that m(z) = dim(C) — Yli=i dim(OJ> where, for i = 1, 2, . . . , 6. 
Ji = oj- 1 (f3(V(Tty))\(3(zj\. We must therefore show that, for each z G V(T), we have 

Ji C i^j for all i, which would prove that k(z) < m(z); and furthermore, when z = z* , we have 
Ji = Ki for all i, which would prove that k(z*) = m(z*). 

So, consider any z G V(T). From Lemma [6721 we see that if is any component of T — z, 
then 

0(V(T^))\^z)Ca(V(T^)) (9) 

Hence, Jj C ETj. Moreover, when z = z* , we always have z* ^ V(T^ ), and so, again by 
Lemma 16721 equality holds in ©, implying that Jj = Ki. □ 



From Propositions 16. 1 1 and 16.41 we obtain the following theorem. 

Theorem 6.5. Let (Q, oj) be a graph decomposition of a code C, and let (T, (3) be a vertex-cut tree 
of Q. Then, there exists a tree decomposition (T, 7) of C such that, for any graphical realization 



CONSTRAINT COMPLEXITY OF GRAPHICAL REALIZATIONS 



17 



T E 9t(C; G, w), we /jave 

k(C;T, 7 ) 



«(r)> 



vc- width (T, 



Hence, re(C;0,w) > ^g^gy - 



The theorem above is a fundamental result with several important consequences, some of which 
we present here. The first of these is a corollary that gives a lower bound on the K-complexity of 
any realization of a given code C on a graph G- 

Corollary 6.6. For a code C and a connected graph G, we have 

k(C;G) > Kaee ^\ > KtrM 



^tree 

Proof. Consider an arbitrary index mapping uj : I — > V(G), where / is the index set of C. Let 
(T, (5) be an optimal vertex-cut tree of G, by which we mean that vc-width(T, (3) = K vc -tree(G)- By 
Theorem [631 there exists a tree decomposition (T, 7) of C such that 

k(C;G,ol>) > > 



vc- width (T, j3) K vc . tree (C7) 
Minimizing over all possible mappings lo : I — > V(G), we obtain 

k{C,G) > j^r. 

™vc-tree ytl ) 

From Lemma [5^21 we also have n vc . tree (G) < fttree(£7) + 1- □ 

If, in the above proof, we take (T, (3) to be an optimal vertex-cut path instead, i.e., take (T, j3) to 
be a vertex-cut path such that vc- width (T, 0) = K V c-path(^)> then we obtain the next corollary. 

Corollary 6.7. For a code C and a connected graph G, we have 

K (C;G)> Kpath(C) > KpMC) 



Kvc-path(^) K path (C7) + 1 

The (first) inequality above can be tight, in the sense that there are examples of codes C and 
graphsgforwhichK(C;g) = r^| y l. 

Example 6.2. Take C to be the [24, 12,8] binary Golay code, and let (5 be the family of graphs 
consisting of all n-cycles. From Example \5.1\ we know that tt V c-path(^) = 2 for any G € (25. It is 
also known that K pat h(C) = 9 (7J Example 11, AT. Section 5]. Hence, by the bound of Corollarv \6.7\ 
noting that k(C; G) must be an integer, we have k(C; G) > 5 for any G € (5. Thus, k(C; (5) > 5. 
The tailbiting trellis realization of the Golay code given in (5 ] has K-complexity equal to 5, from 
which we conclude that k(C; 0) = 5. 

Using Corollary 16.71 as a starting point, we derive a lower bound on the treewidth of an [n, k, d] 
linear code. For this, we will also need the following result @ Theorem 7.1]: if G is a graph with 
treewidth at most I, then the pathwidth of G is at most (£ + 1) log 2 |V"(£7)|. 

Proposition 6.8. For an [n, k, d] linear code C, with n > 1, 

(r , . ^path(C) k(d-l) 



3 + 21og 2 (ra-l) - ra(3 + 21og 2 (n-l))' 



18 



N. KASHYAP 



Proof. The second inequality above is due to the fact, shown in lEl, thaj] for an [n,k,d] linear 
code C, Kp at h(C) > k(d — l)/n. 

The first inequality is proved as follows. Let (T,u) be an optimal tree decomposition of C, so 
that k(C; T, to) = /%ee(C). As mentioned in Section [3l (T, uj) may be chosen so that T is a cubic 
tree, and u is a bijection between the index set of C and the leaves of T. Thus, T is a cubic tree with 
n leaves, from which it follows that T has n — 2 internal nodes. Hence, |V(T)| = 2n — 2. Since T 
has treewidth equal to one, by the result from |4] quoted earlier, K pat h(T) < 2 log 2 (2n — 2). 

We now have, via Corollary 16.71 

k(C;T,u) > k(C;T) > ^ th(C) > k p^( C \ 

K ' ' K > ' ~ Kpath (T) + l " 21og 2 (2n-2) + l' 

which proves the proposition. □ 
In particular, Proposition 16. 8 1 shows that for any linear code C of length n, 

K tree ) 

This estimate of the ratio ^^^j is the best possible^, up to the constant implicit in the O-notation. 

Indeed, it was shown in ifTTTl that a sequence of codes C^' (i = 1,2,3, .. .), with length rii = 
12(2* - 1) + 2, Kfee(C (i) ) = 2 and K path (C w ) > \{i + 3), can be constructed over any finite field 

F. It is clear that ^ palh ^ w j grows logarithmically with codelength n;. 



Ktree 



A less precise formulation of the second inequality in Proposition 16. 81 is also instructive: there 
exists a constant cq > such that, for any [n, k, d] linear code C, with n > 1, 

k d 

Ktree(C) > C — : . (10) 

n log 2 n 

A noteworthy implication of the above inequality is that code families of bounded treewidth are 
not very good from an error-correcting perspective. Given an integer t > 0, denote by TW(i) the 
family of all codes over F of treewidth at most t. A code family £ is called asymptotically good if 
there exists a sequence of [n» 3 hi, dj\ codes G <£, with lira; rtj = oo, such that liminfj ki/rii and 
liminfj di/rii are both strictly positive. The following result, an easy consequence of (flOl . resolves 
a conjecture in ifTTl . 

Corollary 6.9. Lef C®, i = 1, 2, 3, be any sequence of [ra», fcj, rfi] codes such that 

Hm . tree (C«)^ = 0. 

r/jen, either Hindoo /cj/nj = or lim^oo di/rii = 0. In particular, for any t > 0, f/je code family 
TW(t) is not asymptotically good. 

In fact, a more general result is true. For a fixed integer I > 0, let 0^ denote the family of all 
graphs with vc-treewidth at most £. In particular, note that, by Lemma \5?2\ <5g contains all graphs 
with treewidth at most I — 1. Then, for an [n, k, d] linear code C with n > 1, we have, via (O, 
Corollary 16.61 and Proposition [621 



?n(3 + 21og 2 (n- 1)) 



7 The result in 1151 is only explicitly stated as a lower bound on the state max-complexity of any conventional trellis 
realization of C. However, the state max-complexity of a graphical realization can never exceed the constraint max- 
complexity of the realization, as noted in Section[3] 

8 It was conjectured in 1111 that for codes C of length n, n pat h(C) — Ktree(C) = 0(log n), but we now do not believe this 
to be true. 
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We thus have the following corollary to Proposition 16. 8[ which extends Corollary 16.91 

Corollary 6.10. Given an integer t > 0, if £. is a family of codes over ¥ with the property that 
k(C;<3i) < tfor all C € £, then <L is not asymptotically good. 

A rough interpretation of the last result is that a "good" error-correcting code cannot have a low- 
complexity realization on a graph with small (vc-)treewidth. In other words, codes that are good 
from an error-correcting standpoint can have low-complexity realizations only on graphs with large 
(vc-)treewidth. 

7. Concluding Remarks 

In this paper, we demonstrated at length the use of the Vertex-Cut Bound in finding lower bounds 
on the K-complexity of graphical realizations. As suggested at the end of Section [4] a natural appli- 
cation of the Vertex-Cut Bound is to formulate constrained optimization problems whose solutions 
are lower bounds to various measures of constraint complexity. This method could be used, for in- 
stance, to determine lower bounds on the true computational complexity of sum-product decoding 
for a given code. This can be done by choosing a constraint complexity measure that accurately 
reflects the cost of sum-product decoding. Such a complexity measure can be chosen based on 
detailed counts of the number of arithmetic operations required in the sum-product algorithm; see, 
for example, ULEILQ- 

It is now probably fair to say that the main open problem in the area of graphical realizations 
of codes is to explicitly construct, for a given code C and an arbitrary graph Q, realizations of C 
on G whose constraint complexity is within striking distance of the lower bounds found using the 
methods of this paper. While we have shown that our bounds can be tight for specific examples of 
codes C and graphs Q, their tightness in the generic instance remains to be investigated. 

Appendix A. An Example 

We provide here an example of a code C and a graph decomposition (G, u) of C, for which there 
is no realization F G %K(C;G,u) such that k(T) = k(C;G,oj) and « + (r) = n + {C; Q, oj). The 
example we give is based on Example 3.1 in |[T4l . 

Consider the [11, 3, 3] binary linear code C generated by the codewords 00011100000, 00000111000 
and 00101010100. We take I = {1, 2, 3, ... , 11} to be the index set of C, identifying the index % 
with the ith coordinate of C. Now, let G be the 11-cycle, and oj : I — > V(G) the index mapping 
depicted in Figure [T] Figure [6] shows a tailbiting trellis realization, F±, of C that extends (G,oj). 
Note that k(Fi) = 2, while k + (Fi) = 14. A second tailbiting trellis realization, T2, in 9t(C; G,oj) 
is shown in Figure|7J with k(F2) = 3 and k + (F2) = 13. It can be shown (using arguments similar 
to those given in Example 3.1 in lfi4lD that k(F\) = k(C; G,ui), while k + (F2) = k + (C; G,oj), and 
that there is no realization F G 9l(C; G, oj) that simultaneously achieves k(C; G, u>) and k + (C; G, oj). 

References 

[1] S.M. Aji and R.J. McEliece, "The generalized distributive law," IEEE Trans. Inform. Theory, vol. 46, no. 2, pp. 
325-343, 2000. 

[2] S. Arnborg, D.G. Corneil and A. Proskurowski, "Complexity of finding embeddings in a fc-tree," S1AM J. Alg. Disc. 

Meth, vol. 8, pp. 277-284, 1987. 
[3] H.L. Bodlaender, "A tourist guide through treewidth," Acta Cybernetica, vol. 11, pp. 1-23, 1993. 
[4] H.L. Bodlaender, T. Kloks, "Efficient and constructive algorithms for the pathwidth and treewidth of graphs," J. 

Algorithms, vol. 21, no. 2, pp. 358-402, 1996. 
[5] A.R. Calderbank, G.D. Forney Jr., and A. Vardy, "Minimal tail-biting trellises: The Golay code and more," IEEE 

Trans. Inform. Theory, vol. 45, pp. 1435-1455, July 1999. 
[6] G.D. Forney Jr., "Codes on graphs: normal realizations," IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 520-548, 

Feb. 2001. 

[7] G.D. Forney Jr., "Codes on graphs: constraint complexity of cycle-free realizations of linear codes," IEEE Trans. 
Inform. Theory, vol. 49, no. 7, pp. 1597-1610, July 2003. 



20 



N. KASHYAP 



0000000000 
• • • «- » «- » «- -» • • • 



• — ^ — • — - — • 






10101010 

(a) 



[3,1] [3,1] [3,1] [3,2] [3,1] [3,2] [3,1] [3,2] [3,1] [3,1] [3,1] 



(b) 

FIGURE 6. A tailbiting trellis realization for an [11,3,3] binary linear code, 
(a) Trellis diagram, (b) Depiction of trellis realization showing the length and 
dimension of the local constraint codes; all state spaces have dimension 1. 
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Figure 7. A tailbiting trellis realization for an [11,3,3] binary linear code, 
(a) Trellis diagram, (b) Depiction of trellis realization showing the length and 
dimension of the local constraint codes, and the dimensions of the state spaces. 
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